HBench: JGC-An Application-Specific Benchmark Suite for Evaluating JVM Garbage Collector Performance

As Java becomes a viable platform for server applications, performance becomes a greater concern. An important aspect of Java Virtual Machine performance is its dynamic memory management system (garbage collection or GC). Traditional GC benchmarking often focuses on a set of fixed applications. As a result, when an actual application’s memory behavior differs from that of the standard benchmarks, the benchmark results do not help the user judge which GC implementation suits her application the best. In this paper, we present HBench:JGC, an application-specific benchmarking suite, based on the idea that a system’s performance be measured in the context of a specific application. HBench:JGC employs a methodology that characterizes the application memory usage and the GC implementation independently and carefully combines both characterizations to form a single metric that reflects a particular application’s performance in the presence of a particular GC implementation. We evaluate our approach on Sun Microsystems’s JDK1.2.2 classic JVM with a sequential mark-sweep GC. Our results demonstrate HBench:JGC’s unique predictive power and its ability to provide meaningful metrics that lead to a better understanding of GC performance.